The P3 indexes the distance between perceived and target image.

Visual recognition requires inferring the similarity between a perceived object and a mental target. However, a measure of similarity is difficult to determine when it comes to complex stimuli such as faces. Indeed, people may notice someone "looks like" a familiar face, but find it hard to describe on the basis of what features such a comparison is based. Previous work shows that the number of similar visual elements between a face pictogram and a memorized target correlates with the P300 amplitude in the visual evoked potential. Here, we redefine similarity as the distance inferred from a latent space learned using a state-of-the-art generative adversarial neural network (GAN). A rapid serial visual presentation experiment was conducted with oddball images generated at varying distances from the target to determine how P300 amplitude related to GAN-derived distances. The results showed that distance-to-target was monotonically related to the P300, showing perceptual identification was associated with smooth, drifting image similarity. Furthermore, regression modeling indicated that while the P3a and P3b sub-components had distinct responses in location, time, and amplitude, they were similarly related to target distance. The work demonstrates that the P300 indexes the distance between perceived and target image in smooth, natural, and complex visual stimuli and shows that GANs present a novel modeling methodology for studying the relationships between stimuli, perception, and recognition.

Visible skin disease symptoms of another person reduce automatic imitation of their hand movements.

Imitation is an important mechanism for social interaction and learning, and humans tend to imitate others automatically. While imitating others is often useful, it can backfire when imitation is incongruent with one's goals. For example, in forced-choice reaction time tasks, this tendency results in a reliable slowing of reactions if the observed and self-initiated actions are incompatible (compatibility effect). While imitation is commonly explained as a social phenomenon, previous results on the compatibility effect's dependence on social cues are inconsistent. However, in many previous studies, the associated social cues were easy to ignore by the participants. To make the social modifier more salient, the current study manipulated emotionally relevant aspects of the model hand itself in an imitation inhibition task by using models displaying skin disease symptoms which we expected to elicit (1) perceptions of dissimilarity and (2) disgust in participants. As predicted, participants' (n = 63) reaction times were influenced more by the incompatible actions of the symptom-free than the symptomatic model hand. However, both levels of self-reported disgust toward and self-other overlap with symptomatic hands were low and did not account for the observed effect on automatic imitation. Our findings show that automatic imitation depends on social factors if these are an integral part of the model and processed quickly, presumably due to their affective salience or the salience of the self-other distinction. Whether this effect is driven by emotional reactions to the model remains an open question.

Anticipation of aversive visual stimuli lengthens perceived temporal duration.

Subjective estimates of elapsed time are sensitive to the fluctuations in an emotional state. While it is well known that dangerous and threatening situations, such as electric shocks or loud noises, are perceived as lasting longer than safe events, it remains unclear whether anticipating a threatening event speeds up or slows down subjective time and what defines the direction of the distortion. We examined whether the anticipation of uncertain visual aversive events resulted in either underestimation or overestimation of perceived duration. The participants did a temporal bisection task, where they estimated durations of visual cues relative to previously learnt long and short standard durations. The colour of the to-be-timed visual cue signalled either a 50% or 0% probability of encountering an aversive image at the end of the interval. The cue durations were found to be overestimated due to anticipation of aversive images, even when no image was shown afterwards. Moreover, the overestimation was more pronounced in people who reported feeling more anxious while anticipating the image. These results demonstrate that anxiogenic anticipation of uncertain visual threats induce temporal overestimation, which questions a recently proposed view that temporal underestimation evoked by uncertain threats is due to anxiety.

Space, time and number: common coding mechanisms and interactions between domains.

Space, time and number are key dimensions that underlie how we perceive, identify and act within the environment. They are interconnected in our behaviour and brain. In this study, we examined interdependencies between these dimensions. To this end, left- and right-handed participants performed an object collision task that required space-time processing and arithmetic tests that involved number processing. Handedness of the participants influenced collision detection with left-handers being more accurate than right-handers, which is in line with the premise that hand preference guides individual differences as a result of sensorimotor experiences and distinct interhemispheric integration patterns. The data further showed that successful collision detection was a predictor for arithmetic achievement, at least in right-handers. These findings suggest that handedness plays a mediating role in binding information processing across domains, likely due to selective connectivity properties within the sensorimotor system that is guided by hemispheric lateralisation patterns.

Time to imagine moving: Simulated motor activity affects time perception.

Sensing the passage of time is important for countless daily tasks, yet time perception is easily influenced by perception, cognition, and emotion. Mechanistic accounts of time perception have traditionally regarded time perception as part of central cognition. Since proprioception, action execution, and sensorimotor contingencies also affect time perception, perception-action integration theories suggest motor processes are central to the experience of the passage of time. We investigated whether sensory information and motor activity may interactively affect the perception of the passage of time. Two prospective timing tasks involved timing a visual stimulus display conveying optical flow at increasing or decreasing velocity. While doing the timing tasks, participants were instructed to imagine themselves moving at increasing or decreasing speed, independently of the optical flow. In the direct-estimation task, the duration of the visual display was explicitly judged in seconds while in the motor-timing task, participants were asked to keep a constant pace of tapping. The direct-estimation task showed imagining accelerating movement resulted in relative overestimation of time, or time dilation, while decelerating movement elicited relative underestimation, or time compression. In the motor-timing task, imagined accelerating movement also accelerated tapping speed, replicating the time-dilation effect. The experiments show imagined movement affects time perception, suggesting a causal role of simulated motor activity. We argue that imagined movements and optical flow are integrated by temporal unfolding of sensorimotor contingencies. Consequently, as physical time is relative to spatial motion, so too is perception of time relative to imaginary motion.

Neuroadaptive modelling for generating images matching perceptual categories.

Brain-computer interfaces enable active communication and execution of a pre-defined set of commands, such as typing a letter or moving a cursor. However, they have thus far not been able to infer more complex intentions or adapt more complex output based on brain signals. Here, we present neuroadaptive generative modelling, which uses a participant's brain signals as feedback to adapt a boundless generative model and generate new information matching the participant's intentions. We report an experiment validating the paradigm in generating images of human faces. In the experiment, participants were asked to specifically focus on perceptual categories, such as old or young people, while being presented with computer-generated, photorealistic faces with varying visual features. Their EEG signals associated with the images were then used as a feedback signal to update a model of the user's intentions, from which new images were generated using a generative adversarial network. A double-blind follow-up with the participant evaluating the output shows that neuroadaptive modelling can be utilised to produce images matching the perceptual category features. The approach demonstrates brain-based creative augmentation between computers and humans for producing new information matching the human operator's perceptual categories.

Information gain modulates brain activity evoked by reading.

The human brain processes language to optimise efficient communication. Studies have shown extensive evidence that the brain's response to language is affected both by lower-level features, such as word-length and frequency, and syntactic and semantic violations within sentences. However, our understanding on cognitive processes at discourse level remains limited: How does the relationship between words and the wider topic one is reading about affect language processing? We propose an information theoretic model to explain cognitive resourcing. In a study in which participants read sentences from Wikipedia entries, we show information gain, an information theoretic measure that quantifies the specificity of a word given its topic context, modulates word-synchronised brain activity in the EEG. Words with high information gain amplified a slow positive shift in the event related potential. To show that the effect persists for individual and unseen brain responses, we furthermore show that a classifier trained on EEG data can successfully predict information gain from previously unseen EEG. The findings suggest that biological information processing seeks to maximise performance subject to constraints on information capacity.

No evidence of calorie-related modulation of N2 in food-related Go/No-Go training: A preregistered ERP study.

Go/No-Go tasks, which require participants to inhibit automatic responses to images of palatable foods, have shown diagnostic value in quantifying food-related impulses. Moreover, they have shown potential for training to control impulsive eating. To test the hypothesis that training modulates early neural markers of response inhibition, the current study investigated how the N2 event-related brain potential to high- and low-calorie food images changes along Go-/No-Go training and how the N2 is related to later eating behavior. 50 healthy adults, (mBMI  = 23.01) first completed a food Go/No-Go task in which high- and low-calorie food images were accompanied by Go-and No-Go-cues with equal frequency. Participants then completed a training block in which high-calorie foods were predominantly paired with a No-Go cue and the low-calorie foods with a Go cue, followed by a block with reversed coupling (order of the training blocks counterbalanced between participants). After each training, there was a snacking opportunity during which calorie intake was measured. Against our preregistered hypotheses, the N2-amplitudes were not significantly affected by the calorie-content and there was no training-related modulation in the N2. In addition , food intake was not influenced by the preceding training blocks and the N2 amplitude did not predict the food intake. Our study suggests that the link between N2 obtained in a food-related Go/No-Go task and impulse control is not clear-cut and may be limited to specific task characteristics. The results are of high importance as they question the previously assumed mechanism of Go/No-Go training in food-related inhibitory control.

I'll make you an offer you can refuse: How socially sensitive is the MFN?

Medial frontal negativity (MFN) is an event-related potential thought to originate in the anterior cingulate cortex. It is evoked by outcomes being worse than expected, such as when presented with unfair economic proposals during the Ultimatum Game (UG). This could mean the MFN indexes a social-emotional response, as commonly suggested in accounts that relate it to a violation of a social norm of fairness. To examine the link between MFN and norm violation, we designed an EEG experiment with participants acting as representatives in an UG. Participants responded either as themselves, or as representatives of two charities. Of these, a norm-compatible charity conformed to the participant's values, while the norm-incompatible charity contrasted to them. The behavioral results showed that norm-incompatible representation reversed behavior, with almost all fair offers being declined. The MFN, however, was unaffected by the norm representation, with unfair offers consistently evoking MFNs across conditions. We furthermore replicated the curious finding that unexpectedly generous offers evoke as much MFN as unfair offers. Thus, the MFN is not nearly as sensitive to higher-order social-emotional processes as commonly assumed. Instead, the perceived inequality that drives the MFN likely reflects a rational, probabilistic process.

The semiotics of the message and the messenger: How nonverbal communication affects fairness perception.

Nonverbal communication determines much of how we perceive explicit, verbal messages. Facial expressions and social touch, for example, influence affinity and conformity. To understand the interaction between nonverbal and verbal information, we studied how the psychophysiological time-course of semiotics-the decoding of the meaning of a message-is altered by interpersonal touch and facial expressions. A virtual-reality-based economic decision-making game, ultimatum, was used to investigate how participants perceived, and responded to, financial offers of variable levels of fairness. In line with previous studies, unfair offers evoked medial frontal negativity (MFN) within the N2 time window, which has been interpreted as reflecting an emotional reaction to violated social norms. Contrary to this emotional interpretation of the MFN, however, nonverbal signals did not modulate the MFN component, only affecting fairness perception during the P3 component. This suggests that the nonverbal context affects the late, but not the early, stage of fairness perception. We discuss the implications of the semiotics of the message and the messenger as a process by which parallel information sources of "who says what" are integrated in reverse order: of the message, then the messenger.

Motor Timing and Covariation with Time Perception: Investigating the Role of Handedness.

Time is a fundamental dimension of our behavior and enables us to guide our actions and to experience time such as predicting collisions or listening to music. In this study, we investigate the regulation and covariation of motor timing and time perception functions in left- and right-handers who are characterized by distinct brain processing mechanisms for cognitive-motor control. To this purpose, we use a combination of tasks that assess the timed responses during movements and the perception of time intervals. The results showed a positive association across left- and right-handers between movement-driven timing and perceived interval duration when adopting a preferred tempo, suggesting cross-domain coupling between both abilities when an intrinsic timescale is present. Handedness guided motor timing during externally-driven conditions that required cognitive intervention, which specifies the relevance of action expertise for the performance of timed-based motor activities. Overall, our results reveal that individual variation across domain-general and domain-specific levels of organization plays a steering role in how one predicts, perceives and experiences time, which accordingly impacts on cognition and behavior.

Manipulating Bodily Presence Affects Cross-Modal Spatial Attention: A Virtual-Reality-Based ERP Study.

Earlier studies have revealed cross-modal visuo-tactile interactions in endogenous spatial attention. The current research used event-related potentials (ERPs) and virtual reality (VR) to identify how the visual cues of the perceiver's body affect visuo-tactile interaction in endogenous spatial attention and at what point in time the effect takes place. A bimodal oddball task with lateralized tactile and visual stimuli was presented in two VR conditions, one with and one without visible hands, and one VR-free control with hands in view. Participants were required to silently count one type of stimulus and ignore all other stimuli presented in irrelevant modality or location. The presence of hands was found to modulate early and late components of somatosensory and visual evoked potentials. For sensory-perceptual stages, the presence of virtual or real hands was found to amplify attention-related negativity on the somatosensory N140 and cross-modal interaction in somatosensory and visual P200. For postperceptual stages, an amplified N200 component was obtained in somatosensory and visual evoked potentials, indicating increased response inhibition in response to non-target stimuli. The effect of somatosensory, but not visual, N200 enhanced when the virtual hands were present. The findings suggest that bodily presence affects sustained cross-modal spatial attention between vision and touch and that this effect is specifically present in ERPs related to early- and late-sensory processing, as well as response inhibition, but do not affect later attention and memory-related P3 activity. Finally, the experiments provide commeasurable scenarios for the estimation of the signal and noise ratio to quantify effects related to the use of a head mounted display (HMD). However, despite valid a-priori reasons for fearing signal interference due to a HMD, we observed no significant drop in the robustness of our ERP measurements.

Natural brain-information interfaces: Recommending information by relevance inferred from human brain signals.

Finding relevant information from large document collections such as the World Wide Web is a common task in our daily lives. Estimation of a user's interest or search intention is necessary to recommend and retrieve relevant information from these collections. We introduce a brain-information interface used for recommending information by relevance inferred directly from brain signals. In experiments, participants were asked to read Wikipedia documents about a selection of topics while their EEG was recorded. Based on the prediction of word relevance, the individual's search intent was modeled and successfully used for retrieving new relevant documents from the whole English Wikipedia corpus. The results show that the users' interests toward digital content can be modeled from the brain signals evoked by reading. The introduced brain-relevance paradigm enables the recommendation of information without any explicit user interaction and may be applied across diverse information-intensive applications.

Manual dexterity: Functional lateralisation patterns and motor efficiency.

Manual tasks are an important goal-directed ability. In this EEG work, we studied how handedness affects the hemispheric lateralisation patterns during performance of visually-driven movements with either hand. The neural correlates were assessed by means of EEG coherence whereas behavioural output was measured by motor error. The EEG data indicated that left- and right-handers showed distinct recruitment patterns. These involved local interactions between brain regions as well as more widespread associations between brain systems. Despite these differences, brain-behaviour correlations highlighted that motor efficiency depended on left-sided brain regions across groups. These results suggest that skilled hand motor control relies on different neural patterns as a function of handedness whereas behavioural efficiency is linked with the left hemisphere. In conclusion, the present findings add to our understanding about principles of lateralised organisation as a function of handedness.

Not My Problem: Vicarious Conflict Adaptation with Human and Virtual Co-actors.

The Simon effect refers to an incompatibility between stimulus and response locations resulting in a conflict situation and, consequently, slower responses. Like other conflict effects, it is commonly reduced after repetitions, suggesting an executive control ability, which flexibly rewires cognitive processing and adapts to conflict. Interestingly, conflict is not necessarily individually defined: the Social Simon effect refers to a scenario where two people who share a task show a conflict effect where a single person does not. Recent studies showed these observations might converge into what could be called vicarious conflict adaptation, with evidence indicating that observing someone else's conflict may subsequently reduce one's own. While plausible, there is reason for doubt: both the social aspect of the Simon Effect, and the degree to which executive control accounts for the conflict adaptation effect, have become foci of debate in recent studies. Here, we present two experiments that were designed to test the social dimension of the effect by varying the social relationship between the actor and the co-actor. In Experiment 1, participants performed a conflict task with a virtual co-actor, while the actor-observer relationship was manipulated as a function of the similarity between response modalities. In Experiment 2, the same task was performed both with a virtual and with a human co-actor, while heart-rate measurements were taken to measure the impact of observed conflict on autonomous activity. While both experiments replicated the interpersonal conflict adaptation effects, neither showed evidence of the critical social dimension. We consider the findings as demonstrating that vicarious conflict adaptation does not rely on the social relationship between the actor and co-actor.

The self in conflict: actors and agency in the mediated sequential Simon task.

Executive control refers to the ability to withstand interference in order to achieve task goals. The effect of conflict adaptation describes that after experiencing interference, subsequent conflict effects are weaker. However, changes in the source of conflict have been found to disrupt conflict adaptation. Previous studies indicated that this specificity is determined by the degree to which one source causes episodic retrieval of a previous source. A virtual reality version of the Simon task was employed to investigate whether changes in a visual representation of the self would similarly affect conflict adaptation. Participants engaged in a mediated Simon task via 3D "avatar" models that either mirrored the participants' movements, or were presented statically. A retrieval cue was implemented as the identity of the avatar: switching it from a male to a female avatar was expected to disrupt the conflict adaptation effect (CAE). The results show that only in static conditions did the CAE depend on the avatar identity, while in dynamic conditions, changes did not cause disruption. We also explored the effect of conflict and adaptation on the degree of movement made with the task-irrelevant hand and replicated the reaction time pattern. The findings add to earlier studies of source-specific conflict adaptation by showing that a visual representation of the self in action can provide a cue that determines episodic retrieval. Furthermore, the novel paradigm is made openly available to the scientific community and is described in its significance for studies of social cognition, cognitive psychology, and human-computer interaction.

Hemispheric asymmetries and the control of motor sequences.

Sequencing of finger positions reflects a prototype of skilled behaviour. In order to perform sequencing, cognitive control supports the requirements and postural transitions. In this electroencephalography (EEG) study, we evaluate the effects of hand dominance and assess the neural correlates of unimanual and bimanual sequencing in left- and right-handers. The behavioural measurements provided an index of response planning (response time to first key press) and response execution (time between successive key presses, taps/s and percentage of correct responses), whereas the neural dynamics was determined by means of EEG coherence, expressing the functional connectivity between brain areas. Correlations between brain activity and behaviour were calculated for exploring the neural correlates that are functionally relevant for sequencing. Brain-behavioural correlations during response planning and execution revealed the significance of circuitry in the left hemisphere, underlining its significant role in the organisation of goal-directed behaviour. This lateralisation profile was independent of intrinsic constraints (hand dominance) and extrinsic demands (task requirements), suggesting essential higher-order computations in the left hemisphere. Overall, the observations highlight that the left hemisphere is specialised for sequential motor organisation in left- and right-handers, suggesting an endogenous hemispheric asymmetry for compound actions and the representation of skill; processes that can be separated from those that are involved in hand dominance.

The meaning of the virtual Midas touch: an ERP study in economic decision making.

The Midas touch refers to the altruistic effects of a brief touch. Though these effects have often been replicated, they remain poorly understood. We investigate the psychophysiology of the effect using remotely transmitted, precisely timed, tactile messages in an economic decision-making game called Ultimatum. Participants were more likely to accept offers after receiving a remotely transmitted touch. Furthermore, we found distinct effects of touch on event-related potentials evoked by (a) feedback regarding accepted and rejected offers, (b) decision cues related to proposals, and (c) the haptic and auditory cues themselves. In each case, a late positive effect of touch was observed and related to the P3. Given the role of the P3 in memory-related functions, the results indicate an indirect relationship between touch and generosity that relies on memory. This hypothesis was further tested and confirmed in the positive effects of touch on later proposals.

Sequential modulations of the Simon effect depend on episodic retrieval.

Sequential modulations of conflict effects, like the reduction of the Simon effect after incompatible trials, have been taken to reflect the operation of a proactive control mechanism commonly called conflict monitoring. However, such modulations are often contaminated by episodic effects like priming and stimulus-response feature integration. It has previously been observed that if the episodic representation of a conflicting trial is altered by rotating the stimulus framing 180(∘) around its axis, the subsequent "conflict adaptation" pattern is eliminated. In Experiment 1, we replicate the findings and provide the basic episodic interpretation. In Experiment 2, we extend the framework to include rotations of 90(∘), and verify that the episodic effects generalize to scenarios of neutral compatibility. Finally, in Experiment 3, we add complete, 360(∘) rotations, and show that the episodic manipulation by itself does not eliminate the conflict adaptation patterns - as long as conditions favor episodic retrieval. The experiments are argued to demonstrate that an episodic account of the conflict adaptation effect can most parsimoniously account for the behavioral effects without relying on higher order cognition. Accordingly, we conclude that conflict adaptation can be understood either as critically depending on episodic retrieval, or alternatively reflecting only episodic retrieval itself.